The present invention relates to a three-dimensional position and orientation sensing system, and relates, more particularly, to a three-dimensional position and orientation sensing apparatus, a three-dimensional position and orientation sensing method, and a three-dimensional position and orientation sensing system to be used for them, including a computer-readable recording medium, a marker and a probe, for sensing a three-dimensional position and orientation of an object by estimating the three-dimensional position and orientation of the object by the use of an image acquisition apparatus.
In general, the subject of estimating a relative position and orientation between an object and an image acquisition apparatus by recognizing at least three landmarks or markers on the object based on the extraction of these landmarks or markers from the image, is considered a part of an n-point subject, where relative positions of the landmarks are known in advance. (Refer to the literature 1: M. A. Fischler and R. C. Bolles, xe2x80x9cRandom sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,xe2x80x9d Communications of the ACM, Vol. 24, No. 6, June 1981, pp. 381-395.)
In this case, it has been known that when there are only three landmarks, there exist a plurality of solutions.
As a method for solving this problem, there can be pointed out a method as disclosed in Jpn. Pat. Appln. KOKAI Publication No. 7-98208, which utilizes specific markers.
The method disclosed in Jpn. Pat. Appln. KOKAI Publication No. 7-98208 utilizes a positional relationship between one large circle and one small circle.
Further, as another method, there is a system for estimating a three-dimensional position and orientation from an image acquired by a camera by utilizing a plurality of markers of the same shape, as disclosed in the second literature (Refer to the literature 2: W. A. Hoff, T. Lyon, and K. Nguyen, xe2x80x9cComputer Vision-Based Registration Techniques for Augmented Realityxe2x80x9d, Proc. of Intelligent Robots and Computer Vision XV, Lol. 2904, in Intelligent Systems and Advanced Manufacturing, SPIE, Boston, Mass., November 19-21, pp. 538-548, 1996.)
However, according to the technique used in the above-described Jpn. Pat. Appln. KOKAI Publication No. 7-98208, as the markers are basically defined by only one large circle and one small circle defined near this large circle, there are following drawbacks.
(1) When the sizes of the one large circle and one small circle are small respectively in the image, the error of measurement becomes larger.
(2) When it is not possible to recognize the one large circle and one small circle because of occlusion or because of a limit in the image processing, it is not possible to recognize the position and orientation.
Further, according to the above-described literature 2, when a plurality of markers are structured by the same patterns, in many cases, it is difficult to identify the individual markers in many cases.
The identification becomes more difficult when it is not possible to recognize a part of the markers because of occlusion or the like.
Further, when an object is located in a complex environment, there are many cases where there exist other patterns that are similar to those of the markers, when the markers are structured in a single color or in only black or white color. Therefore, it has been difficult to identify the markers from non-marker items.
In the light of the above-described problems, it is an object of the present invention to provide a three-dimensional position and orientation sensing apparatus, a three-dimensional position and orientation sensing method, and a three-dimensional position and orientation sensing system to be used for them, including a computer-readable recording medium, a marker and a probe, which
(1) can estimate the three-dimensional position and orientation of an object, even when a part of markers cannot be observed because of occlusion or the like, and
(2) can estimate the position and orientation from only three markers, by which it has not been possible to achieve by finding a firm solution according to the prior-art n-point subject.
In order to achieve the above object, a first aspect of the present invention provides a three-dimensional position and orientation sensing apparatus comprising:
image input means for inputting an image acquired by an image acquisition apparatus and having at least three markers, three-dimensional positional information of which with respect to an object to be measured is known in advance;
region extracting means for extracting a region corresponding to each marker on the image;
marker identifying means for identifying the individual markers from the characteristics of the appearance of the markers in the extracted regions; and
position and orientation calculating means for calculating a three-dimensional position and orientation of the object to be measured with respect to the image acquisition apparatus, by using positions of the identified markers on the image, and the three-dimensional positional information of the markers with respect to the object to be measured.
Further, a second aspect of the invention provides a three-dimensional position and orientation sensing method for measuring the position and orientation of an object to be measured with respect to an image acquisition apparatus, by analyzing an image acquired by this image acquisition apparatus, the method comprising the steps of:
inputting an image acquired by an image acquisition apparatus and having at least three markers, three-dimensional positional information of which with respect to an object to be measured is known in advance;
extracting a region corresponding to each marker on the image;
identifying the individual markers from the characteristics of the appearance of the markers in the extracted regions; and
calculating a three-dimensional position and orientation of the object to be measured with respect to the image acquisition apparatus, by using positions of the identified markers on the image, and the three-dimensional positional information of the markers with respect to the object to be measured.
Further, a third aspect of the invention provides an article of manufacture comprising a computer-readable recording medium having computer-readable program coding means as a processing program recorded for measuring the position and orientation of an object to be measured with respect to an image acquisition apparatus, by analyzing by computer an image acquired by this image acquisition apparatus, the computer-readable program coding means comprising:
computer-readable programming means for making an image to be input, the image having been acquired by the image acquisition apparatus and having at least three markers, three-dimensional positional information of which with respect to an object to be measured is known in advance;
computer-readable programming means for making an area corresponding to each marker on the image to be extracted;
computer-readable programming means for making the individual markers to be identified from the characteristics of the appearance of the markers in the extracted regions; and
computer-readable programming means for making the three-dimensional position and orientation of the object to be measured with respect to the image acquisition apparatus to be calculated, by using positions of the identified markers on the image, and the three-dimensional positional information of the markers with respect to the object to be measured.
Further, a fourth aspect of the invention provides markers having identification marks disposed on their planes, wherein
the external shapes of the identification marks are circular.
Further, a fifth aspect of the invention provides a probe to be used for measuring a position, the probe comprising:
a contacting portion as a member for contacting an object to be measured; and
a mark portion having identification marks for identifying the probe disposed on the plane of the mark.
Additional objects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objects and advantages of the invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out hereinafter.